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Abstract 


This study conducted in collaboration with a postsecondary institution highlights results 
from a concurrent validity study of administering ACT® section tests to their entering 
freshmen who previously took the ACT test in high school. Students’ ACT scores obtained 
from section retesting were found to be as predictive of first-term grade point average 
(GPA) as scores obtained via traditional ACT testing. Additionally, ACT Superscores 

that were computed across test administrations that included single-subject section test 
events were found to be predictive of first-term GPA, alone and in combination with high 
school GPA. Moreover, the strength of this relationship did not significantly differ from that 
based on students’ most recent ACT Composite scores. 


Introduction 


ACT test scores are designed to measure students’ level of college readiness in key 
core academic content areas (ACT, 2019). Postsecondary institutions use ACT scores in 
combination with other measures such as high school GPA to help inform their admission 
and placement decisions and to identify students most likely to struggle academically, be 
at risk of dropping out, and benefit from institutional services and supports (ACT, 2019; 
Clinedinst, 2019; University of California Academic Senate, 2020). Numerous studies 
have been conducted that provide validity evidence supporting the use of ACT scores for 
these purposes (ACT, 2019; Mattern & Allen, 2016; Radunzel, 2017). 


Beginning in September 2020, three new testing options will be available on national 
ACT test dates: online testing, section retesting, and superscoring. Section retesting 
(also referred to as modular testing or single-subject retesting) gives students the option 
to retake one or more sections of the ACT test instead of having to take the full ACT test 
again. Section retesting will initially only be available to students retesting online. The 
new option of superscoring will allow students who have tested more than once to send 
their ACT Superscore—the average of students’ highest scores in each subject from all 
of their test attempts (including from section retests)—to postsecondary institutions of 
their choice. The option of superscoring is in alignment with current admissions practices 
and policies at many postsecondary institutions and allows students to demonstrate their 
academic achievement most favorably for college applications and scholarships. ' 
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Over the last several years, ACT has been conducting research to examine the validity and 
fairness of different scoring practices and options to help provide insights on how postsecondary 
institutions might best make use of multiple ACT scores when students retest. Results from 

the studies conducted to date support offering the new options of section retesting and 
superscoring. First, in a large multi-institutional study (Mattern, Radunzel, Bertling, & Ho, 2018), 
superscoring was found to be as predictive—if not more predictive—of first-year GPA than 

the other ACT Composite scoring methods examined, which included computing the average 
Composite score across test administrations or using students’ most recent Composite score 

or their highest Composite score; correlations ranged from 0.39 for the average to 0.41 for 
superscoring. The study also found that first-year GPA for students who tested more often was 
underpredicted, but that when examining the prediction accuracy by the number of times tested, 
superscoring resulted in the least amount of prediction error across the four scoring methods. 
These results suggest that ACT subject scores do not have to come from a single test attempt 
to be a valid indicator of students’ college readiness, supporting both superscoring and section 
retesting. 


The Mattern et al. (2018) study also explored the diversity implications for an admitted class of 
using superscores as compared to the other three scoring methods to admit students. Despite 
the fact that underserved students are less likely to retest (Harmston & Crouse, 2016), the 
authors found that superscoring did not result in a less diverse admitted class as compared 

to the other three scoring methods. In a subsequent study (Mattern & Radunzel, 2019), the 
researchers found that superscoring did not exacerbate subgroup differences for the national 
ACT-tested population over those reported based on students’ most recent ACT scores. 


Second, results from a randomized study of 4,000 students conducted in 2016 indicated that 
the order in which the subject tests were administered did not impact student performance 
(Andrews, 2019). More specifically, the study found that students earned subject scores that 
were similar regardless of the order in which the subject tests were taken. Given that ACT 
scores were similar when taken first as compared to the standard position in the full ACT test, 
the findings from this study support the option of offering section retesting where students will 
not have to retake the entire ACT test but can focus their learning efforts on specific subject 
areas of their choice. Despite concerns being raised that section retesting may lead to artificially 
inflated scores, two recent studies (Mattern, Radunzel, & Andrews, 2019; Radunzel & Mattern, 
2020) provide empirical evidence suggesting that this is actually not the case. In particular, 
the results from these two studies demonstrate that students’ performance when retesting in a 
single ACT subject area tends to be consistent with what would be expected based on typical 
test-retest score gains from taking the entire ACT test. 


While decades of research provide evidence that each individual ACT test is a valid and 

reliable measure of students’ college readiness and related to college outcomes (ACT, 2019; 
see chapters 10 and 11), there is a need to examine the predictive validity of section retest 
scores. Moreover, given that the prior study on superscoring (Mattern et al., 2018) was based 
only on full administrations of the ACT, it is of interest to investigate the relationship between 
first-year college outcomes and ACT Superscores that combine scores not only across full 

test administrations but also across section retests. To address these topics, we conducted 

a concurrent validity study in collaboration with a single four-year public university involving 
students from their fall 2019 freshman cohort. In particular, the following two research questions 
were examined in this case study: 
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1. Are section retest scores as predictive of first-term grades as subject scores obtained from 
taking the full ACT test? 


2. Are ACT Superscores that include section retest scores, alone and in combination with high 
school GPA, as predictive of first-term grades as students’ most recent ACT Composite 
scores? 


Data and Methods 
Study Sample 


The participating institution is located in the western region of the country and has a highly 
selective admissions policy. The incoming freshman class size for this institution is a little over 
4,000 students. The institution accepts both ACT and SAT test scores; nearly 60% of students 
submit ACT scores to the institution as part of the admissions process (U.S. Department of 
Education, 2020). 


The institution completed the following activities for the study: (a) assigned a study coordinator to 
serve as the point of contact, (b) recruited first-year, first-time entering domestic college students 
who had previously taken the full ACT test in high school to take a single subject ACT test during 
the first three weeks of the fall 2019 term, (c) administered and proctored single-subject ACT 
tests in paper format in a secure manner and under standard testing conditions, (d) returned 

the completed answer documents to ACT, and (e) submitted a data file of students’ first-term 
grades in January 2020. We aimed to recruit 50 students per ACT subject area (English, math, 
reading, science) for a total of 200 students. Approximately a month before classes began, the 
institution began inviting students to participate via email.” It became apparent early on that it 
would be difficult to achieve this recruitment goal within the narrow study recruitment and testing 
window. For this reason, students were allowed to test in multiple subject areas and to choose the 
subject(s) they wanted to test in. Students who tested in multiple subject areas took each subject 
test on different days to simulate a single-subject test experience. 


Both the institution and study participants were compensated for participating in the study. The 
institution received a monetary incentive for completing the study activities. Students received 

a $50 gift card for each single-section test taken. To increase students’ motivation for testing, 
students were informed that they would receive an additional $50 gift card if they met or exceeded 
their most recent ACT subject score taken during their junior or senior year in high school. 


Complete data was available for a total number of 118 students who had prior ACT scores.® 
The resulting sample size by subject area was 39 in English and math, 50 in reading, and 46 
in science. The institution provided outcomes data on their entire 2019 freshman cohort. There 
were 2,729 non-study participants who had taken the ACT test in high school. This sample of 
ACT-tested nonparticipants was used for comparison purposes. 
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Measures 


Outcome. The primary outcome was the fall-term college GPA, on a scale from 0.00 to 4.00. The 
average first-term GPA for the participant sample was 3.36 with a standard deviation of 0.52. 


Predictors. The following academic achievement measures were included as predictors of 
first-term GPA: ACT section scores in English, math, reading, and science; the ACT Composite 
score; and high school GPA. ACT section scores and the Composite score range from 1 to 36. 
ACT subject scores were either the scores earned on the section tests that were administered 
during the first three weeks of college (labeled as section test) or the scores earned on students’ 
most recent traditional ACT testing event from their junior or senior year in high school (labeled 
as full test-most recent). 


Two different Composite scores were used; these included the most recent ACT Composite 
score earned in high school and the ACT Superscore that was computed by combining the 
highest subject scores across test administrations from a students’ sophomore, junior, and 
senior year in high school, and their section test taken during the first three weeks of college. 
High school GPA was self-reported by students during the ACT registration process and is 
based on their coursework taken in up to 23 specific courses in English, mathematics, social 
studies, and science, and the grades earned in those courses. Prior studies have shown that 
students report high school coursework and grades accurately relative to information provided 
in their high school transcripts (Sanchez & Buddin, 2016). 


Analysis 


Means and percentages were used to describe the outcomes and student characteristics. 
Pearson correlation coefficients were computed between students’ achievement measures 
and first-term GPA. The formulas presented by Steiger (1980) were used to test whether two 
correlations that involve a common outcome variable were significantly different from one 
another. Linear regression models were developed to predict first-term GPA from students’ 
achievement measures. To address study objective 1, analyses were conducted separately for 
each subject sample using the corresponding ACT section score. To address study objective 
2, analyses were conducted for the full participant sample using ACT Composite scores. A 
significance level of .05 was used in this study. 


Description of Samples 


Table 1 provides descriptive information on ACT Composite score, high school GPA, and first- 
term GPA, by sample. On average, students who participated in the study had slightly higher 
ACT Composite scores from high school and earned slightly higher first-term GPAs than ACT- 
tested nonparticipants from the institution. The average high school GPA tended to be more 
comparable across the samples. The average time between when the ACT section test was 
taken as a part of this study and when the full ACT was last taken in high school ranged from 
13.6 months for the English sample to 15.1 months for the science sample. 


As shown in Table A1 in the Appendix, there was representation in the study samples across 
gender, race/ethnicity, socioeconomic status, and initial declared major category, though 
differences existed across samples. For example, the math sample had a higher percentage 

of males and STEM majors than the other samples did (76.9% vs. 43.6% to 56.0% for males 
and 79.5% vs. 54.1% to 67.4% for STEM majors). For each sample, the percentage of students 
returning for the second term was high (ranging from 97.4% to 100.0%). 
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Table 1. Average ACT Composite Score, High School GPA, and First-term GPA by Sample 


ACT-tested 
English SX=¥-Yel Tale} Science raed ay ex-ta iced | eyelet) 


sample sample sample from institution 
WEVar-10){-) n Mean (SD) n Mean(SD) n Mean(SD) n _ Mean(SD) WCF Tam (1 8D) 


ACT Composite 
score — most 39 = 29.6 (3.7) 39 304(42) 50 31.2(3.7) 46 30.6 (4.0) 2,729 27.7 (4.1) 
recent 


Time between 
ACT testing 39 = 13.6 (4.2) 39 14.2 (4.0) 50 14.8(4.5) (46 15.1 (4.8) 
(in months) 


High school GPA 38 3.80(0.19) 37 3.84(0.16) 49 3.79(0.24) 45 3.86(0.15) 2,573 3.80 (0.22) 


First-term GPA 39 3.37(0.43) 39 3.35(0.55) 50 3.45(0.46) 46 3.41(0.52) 2,729 3.16 (0.62) 


Note. GPA = grade point average. SD = standard deviation. Time between ACT testing is the difference in months 
between when the ACT section test was taken as a part of this study and when the full ACT test was last taken in high 
school (labeled as ACT Composite score — most recent). 


Results 
Section Scores 


Table 2 provides average ACT scores for the two testing events by subject and sample, as well 
as the correlations between the two scores. First, looking at the results in the table that are 
labeled as full sample, we see that the average ACT scores on the section retest tended to be 
relatively high, ranging from 27.9 in science to 29.7 in math. However, students’ scores on the 
section retest completed during the first three weeks of college tended to be lower than those 
earned previously in high school when taking the full ACT (by 1.1 score points in math to 2.6 
score points in English). This was seen even after omitting an extreme outlier from the math, 
reading, and science samples.‘ For reference, a study by Harmston and Crouse (2016) found 
that students first testing as juniors demonstrated an average Composite score increase of 1.1 
points by their final ACT test. The magnitude of the average decline in scores may suggest that, 
despite the incentives offered, students were not as motivated on the section retest as they 

had been when taking the full ACT in high school to earn a college-reportable score. In fact, 
compared to their latest test scores from high school, the percentages of students scoring lower 
by more than two standard errors of measurement (SEM) on their section retest ranged from 
16% in reading to 31% in English, which is higher than expected. If students’ true achievement 
remained constant between the two time points, we would have expected fewer than 8% of 
students to score lower by more than two SEM. 


Because some students may not have been fully engaged on the section retest, we also present 
results for a reduced sample that excludes students who experienced large score declines 
(labeled as subsample in the table). Even though around 40% to 50% of students in the 
subsample increased their subject scores on the section retakes, scores tended to be slightly 
lower on the section retest than those earned previously in high school on the full ACT (by 0.2 
point in math to 1.2 points in reading).° Compared to the full sample, the correlations between 
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the full test and section retest scores were higher for the subsample of students (ranging from 
.17 to .94 vs. .65 to .73 for the full sample).° Given that students’ level of motivation could 
significantly impact the results of the study, it is unfortunate that we were unable to quantify and 
therefore control for this factor in the analyses. This is a limitation of this study. 


Table 2. Average ACT Scores and Correlations between ACT Scores by Subject and Sample 


Full test - Difference in Correlations between ACT 
most recent Section test ACT scores scores 
95% 95% 
Subject | Wo)! (=) am © 0) of -1 g 
English 
Full sample 39 31.0 46 28.4 47 -2.6 3.4 mR! 55 85 
Subsample 27 SOS 5.1 29.6 48 -0.7 W/ .94 .88 97 
Math 
Full sample 38 30.8 SU 29.7 3:3 -1.1 2.9 .65 42 .80 
Subsample 31 30.4 3.8 30.2 3.3 -0.2 Bf) AT Ou .88 
Reading 
Full sample 49 3118 44 29.5 a2 -2.4 3.9 .67 A9 .80 
Subsample 42 Sil si 46 30.5 48 -1.2 2s) .88 19 94 
Science 
Full sample 45 29.6 47 27.9 5.0 -1.7 3.6 12 54 84 
Subsample 36 28.8 48 28.5 5.1 -0.4 2.4 .89 19 94 


Note. SD = standard deviation. The full sample includes all students’ scores except the one outlier (see end note #4). 
The subsample excludes students whose scores on section retesting decreased by more than 2 SEM compared to their 
latest full test score from high school. All correlation coefficients were significantly different from 0 (p < .0001). Students 
completed the entire ACT test during their junior or senior year in high school, while students completed the section 
retest during the first three weeks of their freshman year in college. 


Figure 1 provides the predicted first-term GPA as a function of ACT subject score by testing 
event for the subsamples. Figure A1 in the Appendix shows the corresponding figures for the full 
samples, and Table A2 in the Appendix provides the regression estimates by sample and testing 
event. As shown in Figure 1 and Table A2, ACT single-section test scores were predictive 

of first-term GPA among the subsamples of students who may have been more engaged. 
Moreover, in English and reading, the regression lines and parameter estimates associated with 
section retest scores were similar to those estimated using students’ prior full ACT scores. In 
math and science, the slope appeared to be slightly steeper for the section retest scores than 
for the prior full ACT scores. However, the differences in the predicted values between the two 
scores were relatively small (at most 0.13 in math and at most 0.07 in science).’ Moreover, as 
shown in Table A2, the 95% confidence intervals for the slopes associated with standardized 
test scores overlapped between the two testing events, suggesting that the two slopes did not 
significantly differ from one another. Similar conclusions were reached for the full sample (see 
Figure A1 and Table A2). 
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Figure 1. Predicted First-Term GPA by ACT Section Score and Testing Event for Subsamples 
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Table 3 provides the correlation coefficients between ACT subject scores and first-term college 
GPA by subject and sample. For the full sample, the correlation coefficient between the section 
retest score and first-term college GPA was significantly different from zero in each subject area 
(as evidenced by the 95% confidence intervals not including zero); the correlations ranged from 
.31 in reading to .53 in math. This result was also seen for the subsample except in reading 
where the correlation coefficient was nearly significantly different from zero (95% confidence 
interval = -.02 to .54); the correlations ranged from .29 in reading to .52 in math. The test- 
criterion correlations for the section retest scores did not significantly differ from those estimated 
using students’ latest full ACT scores from high school (each p-value greater than .19). 
Moreover, the estimated section retest correlations with first-term GPA are in line with those 
reported in other studies between ACT Composite score and first-year college GPA (Mattern 

et al., 2018; Sawyer, 2010; Westrick, Le, Robbins, Radunzel, & Schmidt, 2015). A possible 
explanation for the correlation coefficient and estimated slope being higher in math than in the 
other subject areas is that there was a higher percentage of STEM majors among the sample 
of students taking the math section test than among the other samples (79.5% vs. 58.0% to 
67.4%, Table A1 in Appendix). 


These results indicate that ACT scores earned in a modular setting are predictive of first-term 
GPA and provide a valid indicator of college readiness. In the next section, we examine the 
validity of using ACT Superscores that include students’ section retest scores in its computation 
to predict first-term GPA, alone and in combination with high school GPA. 
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Table 3. Correlations (R) between ACT Scores and First-Term College GPA by Subject and 
Sample 


Full test — most recent Section test 
95% 95% 95% 95% 
Subject R lower upper lower upper p-value’ 

English 

Full sample 39 34 .03 .59 42 12 {65 486 

Subsample 27 44 .07 10 44 .07 10 .954 
Math 

Full sample 38 45 ails .67 53 .26 Pg) 481 

Subsample 31 42 .O7 .67 202 .20 14 346 
Reading 

Full sample 49 L285 -.03 50 31 .04 95 .576 

Subsample 42 .29 -.02 .54 .29 -.02 54 .990 
Science 

Full sample 45 Foil .02 p55 36 .08 59 .619 

Subsample 36 .30 -.03 so 40 .08 .64 .198 


Note. GPA = grade point average. 95% lower and 95% upper corresponds to the lower and upper limits for the 95% 
confidence intervals. The full sample includes all students’ scores except the one outlier (see endnote #4). The 
subsample excludes students whose scores on section retesting decreased by more than 2 SEM compared to their 
latest full test score from high school. Students completed the entire ACT during their junior or senior year in high 
school, while students completed the section retest during the first three weeks of their freshman year in college. 


1 p-value corresponds to testing whether the two correlations significantly differed from one another (Steiger, 1980). 


ACT Superscores 


Nearly three-fourths of all students in the study sample (74.6%) took the ACT test more than 
once in high school; the average number of times tested with the full test was 2.3 times (not 
including the section retest taken during the first three weeks of college). ACT Superscores 
were computed by combining the highest section scores across test administrations from a 
students’ sophomore, junior, and senior year in high school and their single-section retest. 
Superscores for nearly three-fourths of students were based solely on scores earned when 
completing the ACT during high school. For the remaining 26.3% of the students, the score 
earned on one of the section retakes was the highest section score across test administrations 
and was utilized in computing the ACT Superscore.® As shown in Table 4, the average ACT 
Superscore was slightly higher than the average of students’ most recent ACT Composite 
scores from high school (by 0.7 point, 31.0 vs. 30.3, respectively). This difference in average 
scores between these two scoring methods is consistent with that reported in other studies 
(Mattern & Radunzel, 2019; Mattern et al., 2018). The two Composite scores were highly 
correlated (R = .97; 95% confidence interval = .95, .98). 
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Table 4. Summary of the Relationship with First-Term GPA by Predictor 


PAN OM ME Oxo) an) oLey-J 1 (-M-Lere) c=) 


Statistic Full test — most recent ACT Superscore* High school GPA 
n 118 118 114 
Mean (SD) 30.3 (3.9) 31.0 (3.6) 3.82 (0.21) 


Relationship with first-term GPA 


Regression estimates 


Intercept** 3.36 (3.27, 3.45) 3.36 (3.27, 3.45) 3.35 (3.26, 3.44) 
Slope** 0.18 (0.09, 0.27) 0.16 (0.07, 0.25) 0.15 (0.06, 0.24) 
Correlation** 34 (.17, .49) 32 (.14, .47) 29 (.11, .45) 


Note. GPA = grade point average. SD = standard deviation. For the regression estimates, the predictors were 
standardized to have a mean of 0 and a standard deviation of 1. * ACT Superscore computed by combining the highest 
subject scores across test administrations. ** The values reported include the estimate and the 95% confidence interval 
of the estimate in parentheses. 

As shown in Figure 2 and Table 4, ACT Superscores were found to be predictive of first-term 
GPA. The regression lines and parameter estimates for ACT Superscores were similar to those 
obtained from using students’ most recent ACT Composite score. This is evident by the 95% 
confidence intervals for the slopes associated with the standardized scores overlapping for 

the two scoring methods. Additionally, there was not a significant difference in the correlation 
coefficients between the two scoring methods (Table 4; R = .34 for most recent and .32 for 
Superscore; p-value = .205). These correlations are consistent with that computed for the 
institution’s entire freshman cohort based on students’ most recent ACT Composite score 
earned while in high school (R = .30, 95% confidence interval = .26, .33). 


Figure 2. Predicted First-Term GPA by ACT Composite Score and Scoring Method 
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To further examine the predictive validity of using ACT Superscores, we developed regression 
models that included both ACT scores and high school GPA to estimate students’ first-term 
GPA. According to the single-predictor model shown in Table 4, high school GPA was correlated 
with first-term GPA for the participant sample (R = .29; 95% confidence interval = .11, .45). 
When including both ACT scores and high school GPA in the model (results shown in Table 5), 
ACT Superscore continued to contribute to the prediction of first-term GPA above and beyond 
high school GPA. Specifically, ACT Superscore was found to be significantly related to first- 
term GPA in the joint model that included high school GPA (standardized coefficient = 0.14; 
95% confidence interval = 0.05, 0.23). Additionally, ACT Superscore improved the prediction; 
there was an increase in the validity coefficient associated with the joint model as compared to 
the model that only included high school GPA (R = .40 vs. .29, respectively). Moreover, when 
results for ACT Superscore were compared to those obtained when students’ most recent ACT 
Composite scores were used instead, we found the parameter estimates, multiple correlation 
coefficient, and increase in multiple correlation over high school GPA alone to be similar for the 
two ACT Composite scoring methods (Table 5). The two multiple correlations reported in Table 
5 for the study sample are consistent with that computed for the institution’s entire freshman 
cohort using students’ most recent ACT Composite score from high school in combination with 
high school GPA (R = .42, 95% confidence interval = .39, .45). 


Table 5. Summary of Joint Prediction Models for First-Term GPA by ACT Composite Scoring 
Method 


ACT Composite score 


SELICLI Ce Full test —- most recent ACT Superscore* 


Regression estimates 


Intercept** 3.35 (3.27, 3.44) 3.35 (3.27, 3.44) 

ACT score slope** 0.16 (0.07, 0.25) 0.14 (0.05, 0.23) 

High school GPA slope** 0.12 (0.03, 0.21) 0.13 (0.04, 0.22) 
Multiple Correlation** A2 (.26, .56) 40 (.23, .54) 

Increase in correlation over high 

school GPA alone 0.13 O71 


Note. Model based on 114 participating students with a high school GPA available. For the regression estimates, the 
predictors were standardized to have a mean of 0 and a standard deviation of 1. * ACT Superscore computed by 
combining the highest subject scores across test administrations. ** The values reported include the estimate and the 
95% confidence interval of the estimate in parentheses. 
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Conclusions 


In conclusion, this study provides additional support for the use of section test scores and ACT 
Superscores for predicting students’ first-year college grades. More specifically, findings from this 
study indicate that ACT scores earned from section retests are as predictive of first-term grades 
as scores earned from taking the full test. Additionally, results from the study suggest that ACT 
Superscores obtained from combining scores across test administrations including section retests 
are as predictive of first-term college grades as students’ most recent full test scores. 


Having said that, we acknowledge that this study has limitations. First, this study was conducted in 
collaboration with only one postsecondary institution. Second, the number of participating students 
was relatively small overall and by subject area. This resulted in wide confidence intervals for the 
correlation coefficients and the standardized ACT slope estimates. But, the regression results 

for the sample were found to be similar to those based on the institution’s entire freshman 

cohort of ACT-tested students, suggesting that we would have found similar results if a larger 
number of students had participated from the institution. Moreover, the results from this study are 
consistent with those from an earlier multi-institutional study by Mattern et al. (2018) that found 
ACT Superscores obtained by combining the highest section scores across test administrations to 
be as predictive of first-year grades as other ACT Composite scoring methods. The earlier study 
involved nearly 278,000 students attending 221 four-year institutions. 


Because the section retest occurred during the first three weeks of the fall term and was a 
low-stakes testing event, it was somewhat expected that some students may not have been as 
motivated or prepared for the section retest as they were when they took the full ACT test to 
receive a college-reportable score and used it to gain admissions to colleges. To try to increase 
participant motivation, we offered an extra monetary incentive to students who met or exceeded 
their prior scores from high school. Additionally, analyses were conducted not only on the full 
sample of participants but also on a subsample of students who did not experience a large 
score decline on their section retest as compared to their latest full ACT score from high school. 
The finding that section retest scores are as predictive of first-term grades as scores obtained 
via traditional full ACT testing was not dependent on the sample used in the analyses (full 
sample vs. subsample). Unfortunately, we were unable to quantify and control for students’ level 
of motivation in this study which could have impacted some of the results. Additionally, the small 
sample size prevented examination of differential validity by test administration (section vs. full 
test) for demographic subgroups. Future studies will examine this topic. 


Despite these limitations, the results based on this study suggest that section retest scores 

and ACT Superscores that combine scores across test administrations including section retest 
events are as predictive of first-term college grades as students’ most recent full test scores. 
Once the section retesting and superscoring options become operational in September 2020, 
ACT will work with interested institutions to reexamine these issues. ACT is committed to 
continuing its efforts in conducting national validity studies to provide evidence supporting the 
use of ACT section retest scores and ACT Superscores, in combination with other measures, for 
college admission and course placement decisions and for identification of students who may 
benefit from additional academic services and supports once they matriculate to college. 
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Notes 


1. 


ACT Superscores are typically higher than an ACT Composite score earned from any single test 
attempt (Cruce & Mattern, 2020). 


Initially, study eligibility required that a student be enrolled in a specific subject-relevant course 
to take the corresponding section test (e.g., English Composition | to take the ACT English 
section test). The reason for this requirement was so that we could examine section test scores 
in relation to the grades earned in typical first-year subject-relevant courses. However, this 
eligibility requirement was found to be too restrictive as too few students from the selected 
classes were interested in participating in the study. During the first week of classes, the decision 
was made to open participation in the study to any first-time entering student who had previously 
taken the ACT test in high school during the 2017-18 or 2018-19 academic year. 


Two-thirds of the participants (or 67.0%) took only one single-subject section test; 33.0% took 
multiple section tests (22.9% in two subjects, 5.9% in three subjects, and 4.2% in all four subject 
areas). 


For the math, reading, and science samples, there was one major outlier that scored 12 to 19 
points lower on their section retest as compared to their latest score from high school. The 
corresponding student was omitted from the subject-specific analyses as the inclusion of their 
scores considerably elevated the correlation between ACT section scores and first-term GPA, 
especially in math. If the outlier was included in the subject sample, the correlations for the most 
recent scores and section retest scores were .47 and .62 in math, .27 and .38 in reading, and 
.30 and .38 in science, respectively. 


The percentage of students scoring the same or higher on the section retest as compared to 
their ACT test score from high school was 37% (English), 48% (math), 38% (reading), and 53% 
(science). 


For every subject except math, the correlations between the section retest scores and the full 
test-most recent test scores for the subsample were estimated to be higher than those obtained 
in an earlier study (Radunzel & Mattern, 2020; Table 2). We also found this to be the case when 
the correlations between the section retest scores and the full test-most recent test scores were 
compared to the test-retest Pearson correlation coefficients for the institution’s entire freshman 
cohort who took the full ACT more than once in high school (54.2% of the institution’s ACT-tested 
sample). For the test-retest correlations, a student’s most recent score from high school was 
compared to their closest prior ACT test score; the typical time between these two testing events 
was 3.7 months. The test-retest correlation (and 95% confidence interval) was .74 (.72, .76) in 
English, .80 (.78, .82) in math, .67 (.64, .70) in reading, and .62 (.59, .65) in science. In English, 
reading, and science, the correlations between the section retest scores and full test-most recent 
scores were more similar to the high school test-retest correlations for the full sample than for 
the subsample. In math, the correlation between the section retest scores and full test-most 
recent scores was more similar to the high school test-retest correlation for the subsample than 
for the full sample. 


In math and science, the slopes for the section test scores could be slightly higher than those for 
the full test scores from high school due to the scores being obtained more proximal to when the 
college courses were taken and earned. But, this could also be an artifact of the small sample 
size. 


The corresponding percentages by subject area were 3.4% in English, 9.3% in math, 5.1% in 
reading, and 11.0% in science. These percentages do not sum to the total percentage because 
students could have increased their scores on the section retest in multiple subject areas. 
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Appendix 


Table A1. Percentages of Students by Attribute and Sample 


ACT-tested 
English Math INX=¥- Tel Lare | Science nonparticipants from 
AYETa kel) (=) sample sample sample sample institution 

Sample size 39 39 50 46 2,729 
Gender 

Female 56.4 23.1 44.0 47.8 51.8 

Male 43.6 76.9 56.0 52.2 48.2 
Race/ethnicity 

Underserved minority 30.8 15.4 10.0 19.6 17.9 

White 48.7 46.2 64.0 50.0 59.1 

Asian 12:8 28.2 14.0 17.4 135 

Multiple/unknown race Ta 10.3 12.0 13.0 95 
Annual family income 

Less than $36,000 12.8 12.8 4.0 8.7 4.7 

$36,000 to $80,000 18.0 15.4 14.0 19.6 11.1 

More than $80,000 43.6 38.5 42.0 37.0 49.4 

Missing 25.6 33.3 40.0 34.8 34.8 
Educational aspirations 

Below bachelor’s degree 0.0 0.0 0.0 0.0 0.3 

Bachelor’s degree 35.9 28.2 34.0 23.9 34.2 

Beyond bachelor’s degree 56.4 59.0 54.0 65.2 54.5 

Missing ef 12.8 12.0 10.9 11.0 
Declared major category 

Non-STEM 333 12.8 30.0 15.2 38.3 

STEM 59.0 79.5 58.0 67.4 54.1 

Undecided/unknown Tat Tht 12.0 17.4 tt 
Returned for second term 

Yes 100.0 97.4 100.0 100.0 99.4 

No 0.0 2.6 0.0 0.0 0.6 


Note. Student characteristics provided by institution or obtained from the ACT student record. Underserved minority 
students included African American, American Indian, Hispanic, and Native Hawaiian or other Pacific Islander. Science, 
technology, engineering, and mathematics (STEM) majors included science majors, computer science & mathematics 
majors, engineering & technology majors, and medical & health majors. 
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Table A2. Linear Regression Parameter Estimates for Predicting First-Term GPA by Subject, 
Sample, and Testing Event 


Full test — most recent Section test 
95% 95% 95% 95% 
Subject Predictor : i) = Lower Upper : i) = Lower Upper 
English 
Full sample Intercept 3.37 0.07 3.24 3:51 S07 0.06 3.24 3.50 
ACT score 0.15 0.07 0.01 0.28 0.18 0.06 0.05 0.31 
Subsample Intercept 3.39 0.08 3.23 3.55 3.39 0.08 3:23 3.56 
ACT score 0.20 0.08 0.03 0.37 0.20 0.08 0.03 0.37 
Math 
Full sample Intercept 3.39 0.08 3.24 3.54 3.39 0.07 3.24 3:53 
ACT score 0.23 0.08 0.08 0.39 0.28 0.07 0.13 0.43 
Subsample Intercept 3.40 0.09 3:22 3.58 3.40 0.08 3:23 S08 
ACT score 0.22 0.09 0.04 0.40 0.28 0.08 0.10 0.45 
Reading 
Full sample Intercept 3.46 0.06 3.34 3.59 3.46 0.06 3.34 3.59 
ACT score 0.11 0.06 -0.02 0.24 0.14 0.06 0.02 0.27 
Subsample Intercept 3.51 0.07 3.38 3.64 3.51 0.07 3.38 3.64 
ACT score 0.12 0.07 -0.01 0.26 0.13 0.07 -0.01 0.26 
Science 
Full sample Intercept 3.41 0.07 3.26 3.56 3.42 0.07 Sor 3.56 
ACT score 0.16 0.07 0.01 0.31 0.19 0.07 0.04 0.34 
Subsample Intercept 3.40 0.09 3.23 3.08 3.41 0.08 3.24 3.58 
ACT score 0.16 0.09 -0.02 0.34 0.21 0.08 0.04 0.39 


Note. GPA = grade point average. Est. = estimate. SE = standard error. 95% Lower and 95% Upper corresponds to 

the lower and upper limits for the 95% confidence intervals. ACT subject scores were standardized to have a mean of 

0 and a standard deviation of 1. The full sample includes all students’ scores except the one outlier (see endnote #4). 
The subsample excludes students whose scores on section retesting decreased by more than 2 SEM compared to their 
latest full test score from high school. Students completed the entire ACT test during their junior or senior year in high 
school, while students completed the section retest during the first three weeks of their freshman year in college. 
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Figure A1. Predicted First-Term GPA by ACT Section Score and Testing Event for Full Samples 
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